Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: tree hook for persisting blocks #9365

Merged
merged 20 commits into from
Jul 11, 2024
Merged

feat: tree hook for persisting blocks #9365

merged 20 commits into from
Jul 11, 2024

Conversation

fgimenez
Copy link
Member

@fgimenez fgimenez commented Jul 8, 2024 •
edited
Loading

Closes: #9235

Base automatically changed from fgimenez/tree-run-method to main July 8, 2024 10:28
@fgimenez fgimenez force-pushed the fgimenez/persisting-blocks-tree-hook branch 2 times, most recently from be1d86b to 776bf7a Compare July 9, 2024 10:41
fgimenez
fgimenez commented Jul 10, 2024
View reviewed changes
crates/engine/tree/src/persistence.rs Outdated
@@ -204,12 +204,14 @@ pub enum PersistenceAction {
pub struct PersistenceHandle {
/// The channel used to communicate with the persistence task
sender: Sender<PersistenceAction>,
/// The last persisted block number.
last_persisted_block_number: u64,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes PersistenceHandle stateful (and the methods that modify this field should receive &mut self) let me know wdyt, maybe better to store it in the tree?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think so?

crates/engine/tree/src/tree/mod.rs Outdated
@@ -342,9 +366,38 @@ where
}
}
}

if self.should_persist() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplest possible hardcoded hook, should we use instead something similar to https://github.com/paradigmxyz/reth/tree/main/crates/consensus/beacon/src/engine/hooks ?

crates/engine/tree/src/tree/mod.rs Show resolved Hide resolved
@fgimenez fgimenez marked this pull request as ready for review July 10, 2024 18:09
fgimenez
fgimenez commented Jul 10, 2024
View reviewed changes
crates/engine/tree/src/tree/mod.rs

/// Returns the maximum block number stored.
pub(crate) fn max_block_number(&self) -> BlockNumber {
*self.blocks_by_number.last_key_value().unwrap_or((&BlockNumber::default(), &vec![])).0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar approach as in get_blocks_to_persist, assuming blocks_by_number is the source of truth for blocks to write to db, let me know if that's ok

@mattsse mattsse added C-enhancement New feature or request A-blockchain-tree Related to sidechains, reorgs and pending blocks labels Jul 11, 2024
@fgimenez fgimenez force-pushed the fgimenez/persisting-blocks-tree-hook branch from 81f40bd to c081a04 Compare July 11, 2024 08:59
@fgimenez
Copy link
Member Author

also added logic to remove blocks from memory after successfully persisted

mattsse
mattsse requested changes Jul 11, 2024
View reviewed changes
crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
mattsse
mattsse requested changes Jul 11, 2024
View reviewed changes
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should work

left a few more suggestions/nits, other than that this is what we want here

crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
crates/engine/tree/src/tree/mod.rs Show resolved Hide resolved
crates/engine/tree/src/persistence.rs Outdated Show resolved Hide resolved
crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
mattsse
mattsse approved these changes Jul 11, 2024
View reviewed changes
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, one smol suggestion to use is_some as in_progress

pending @Rjected

crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
Rjected
Rjected approved these changes Jul 11, 2024
View reviewed changes
Copy link
Member

@Rjected Rjected left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one comment about the PersistenceState options

crates/engine/tree/src/tree/mod.rs Outdated Show resolved Hide resolved
@fgimenez fgimenez added this pull request to the merge queue Jul 11, 2024
Merged via the queue into main with commit 11c5e31 Jul 11, 2024
33 checks passed
@fgimenez fgimenez deleted the fgimenez/persisting-blocks-tree-hook branch July 11, 2024 18:01
@fgimenez fgimenez mentioned this pull request Jul 11, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rjected Rjected Rjected approved these changes

@mattsse mattsse mattsse approved these changes

@rkrasiuk rkrasiuk Awaiting requested review from rkrasiuk rkrasiuk is a code owner

@gakonst gakonst Awaiting requested review from gakonst gakonst is a code owner

Assignees
No one assigned
Labels
A-blockchain-tree Related to sidechains, reorgs and pending blocks C-enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Tree hook for persisting blocks
4 participants
@fgimenez @Rjected @gakonst @mattsse